Bi-Touch: Bimanual Tactile Manipulation with Sim-to-Real Deep Reinforcement Learning

Bimanual manipulation with tactile feedback will be key to human-level robot dexterity. However, this topic is less explored than single-arm settings, partly due the availability of suitable hardware along complexity designing effective controllers for tasks relatively large state-action spaces. Here we introduce a dual-arm robotic system (Bi-Touch) based on Tactile Gym 2.0 setup that integrates two affordable industrial-level arms low-cost high-resolution sensors (TacTips). We present suite bimanual tailored towards feedback: bi-pushing, bi-reorienting and bi-gathering. To learn policies, appropriate reward functions these propose novel goal-update mechanism deep reinforcement learning. also apply policies real-world settings sim-to-real approach. Our analysis highlights addresses some challenges met during application, e.g. learned policy tended squeeze an object in task gap. Finally, demonstrate generalizability robustness by experimenting different unseen objects applied perturbations real world. Code videos are available at https://sites.google.com/view/bi-touch/ .